File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\\n'):
                if line:
                    parts = line.split('\\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

Name: T S Rameshkumar <rameshsv06@gmail.com>

Batch: WiproNGA_Datacentre_B9_25VID2182

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

Author: T S Rameshkumar
Email: rameshsv06@gmail.com

Batch: WiproNGA_Datacentre_B9_25VID2182
Program: Wipro NGA Datacentre

© 2025 T S Rameshkumar | File Systems — Storage Systems and Data Management Guide 1.0 documentation

• File Systems
• View page source

---

File Systems

Understanding File Systems

A file system is a method used by operating systems to store, organize, and manage files on storage devices. It defines how data is stored, accessed, and organized on disks.

What is a File System?

A file system provides:

• File Organization: Hierarchical structure for organizing files and directories

• Metadata Management: Information about files (size, permissions, timestamps)

• Space Management: Allocation and deallocation of disk space

• Access Control: Security and permission mechanisms

• Data Integrity: Protection against data corruption

File System Components

Core Components

1. Superblock: Contains metadata about the filesystem

2. Inode Table: Stores file metadata and pointers to data blocks

3. Data Blocks: Actual file content storage

4. Directory Structure: Hierarchical organization of files and folders

5. Journal: Transaction log for filesystem changes (in journaling filesystems)

File System Structure

Filesystem Layout
├── Superblock (filesystem metadata)
├── Group Descriptors (block group information)
├── Block Bitmap (free/used block tracking)
├── Inode Bitmap (free/used inode tracking)
├── Inode Table (file metadata)
└── Data Blocks (actual file content)

Common File Systems in Ubuntu 22.04

ext4 (Fourth Extended Filesystem)

The default filesystem for Ubuntu 22.04:

Features: * Journaling for data integrity * Large file and filesystem support (up to 1 EB) * Backward compatibility with ext2/ext3 * Online defragmentation * Delayed allocation

Use Cases: * General purpose computing * Desktop and server installations * Boot partitions * Home directories

# Create ext4 filesystem
sudo mkfs.ext4 /dev/sdb1

# Check ext4 filesystem
sudo fsck.ext4 /dev/sdb1

# Get ext4 filesystem information
sudo tune2fs -l /dev/sdb1

# Optimize ext4 filesystem
sudo tune2fs -o journal_data_writeback /dev/sdb1

XFS (eXtended File System)

High-performance 64-bit journaling filesystem:

Features: * Excellent scalability * Online resizing (grow only) * Advanced quota management * Allocation groups for parallel I/O * Metadata journaling

Use Cases: * Large files and databases * High-performance computing * Video editing and media storage * Enterprise storage systems

# Create XFS filesystem
sudo mkfs.xfs /dev/sdb1

# Check XFS filesystem
sudo xfs_check /dev/sdb1

# Repair XFS filesystem
sudo xfs_repair /dev/sdb1

# Get XFS information
sudo xfs_info /dev/sdb1

# Resize XFS filesystem (grow only)
sudo xfs_growfs /mnt/xfs

Btrfs (B-tree File System)

Modern copy-on-write filesystem:

Features: * Snapshots and cloning * Built-in RAID support * Compression (zlib, lzo, zstd) * Checksumming for data integrity * Online resizing (grow and shrink)

Use Cases: * System snapshots * Development environments * Data deduplication scenarios * Advanced storage management

# Create Btrfs filesystem
sudo mkfs.btrfs /dev/sdb1

# Mount with compression
sudo mount -o compress=zstd /dev/sdb1 /mnt/btrfs

# Create snapshot
sudo btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot

# List subvolumes
sudo btrfs subvolume list /mnt/btrfs

ZFS (Zettabyte File System)

Advanced filesystem with built-in volume management:

Features: * Built-in RAID (RAID-Z) * Snapshots and clones * Data deduplication * Compression * End-to-end checksumming

Installation and Use:

# Install ZFS on Ubuntu 22.04
sudo apt update
sudo apt install zfsutils-linux

# Create ZFS pool
sudo zpool create mypool /dev/sdb

# Create ZFS dataset
sudo zfs create mypool/data

# Enable compression
sudo zfs set compression=lz4 mypool/data

# Create snapshot
sudo zfs snapshot mypool/data@snapshot1

FAT32 and NTFS

FAT32: Universal compatibility, limited file size (4GB max) NTFS: Windows filesystem with advanced features

# Create FAT32 filesystem
sudo mkfs.fat -F32 /dev/sdb1

# Create NTFS filesystem
sudo mkfs.ntfs /dev/sdb1

# Mount NTFS with full permissions
sudo mount -t ntfs-3g /dev/sdb1 /mnt/ntfs -o permissions

File System Operations in Ubuntu 22.04

Creating File Systems

# ext4 with custom options
sudo mkfs.ext4 -L "MyData" -m 1 /dev/sdb1

# XFS with custom block size
sudo mkfs.xfs -f -b size=4096 /dev/sdb1

# Btrfs with multiple devices (RAID)
sudo mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1

# Set filesystem label
sudo e2label /dev/sdb1 "DataDisk"

Mounting File Systems

# Basic mounting
sudo mount /dev/sdb1 /mnt/data

# Mount with specific options
sudo mount -o rw,noatime,discard /dev/sdb1 /mnt/data

# Mount by UUID (preferred method)
sudo mount UUID=12345678-1234-1234-1234-123456789abc /mnt/data

# Mount with user permissions
sudo mount -o uid=1000,gid=1000 /dev/sdb1 /mnt/data

Checking and Repairing File Systems

# Check filesystem (read-only)
sudo fsck -n /dev/sdb1

# Automatic repair
sudo fsck -p /dev/sdb1

# Force check and repair
sudo fsck -f /dev/sdb1

# Check specific filesystem types
sudo fsck.ext4 /dev/sdb1
sudo xfs_check /dev/sdb1
sudo btrfs check /dev/sdb1

File System Monitoring

Monitoring Tools and Commands

# Real-time filesystem usage
watch df -h

# Inode usage
df -i

# Detailed filesystem information
stat -f /

# Monitor filesystem I/O
sudo iotop -o

# Check filesystem fragmentation (ext4)
sudo e4defrag -c /dev/sdb1

# Btrfs filesystem usage
sudo btrfs filesystem usage /mnt/btrfs

Frequently Asked Questions

Q: Which filesystem should I choose for Ubuntu 22.04?

A: Filesystem selection depends on your use case:

• ext4: Best general-purpose choice, default for Ubuntu

• XFS: Large files, databases, high-performance needs

• Btrfs: Need snapshots, compression, or advanced features

• ZFS: Enterprise features, data integrity critical

# For most users (recommended)
sudo mkfs.ext4 /dev/sdb1

# For large files and databases
sudo mkfs.xfs /dev/sdb1

# For snapshots and modern features
sudo mkfs.btrfs /dev/sdb1

Q: How do I convert between filesystems?

A: Filesystem conversion requires backup and restore:

# 1. Backup data
sudo rsync -av /mnt/source/ /backup/

# 2. Unmount and recreate filesystem
sudo umount /mnt/source
sudo mkfs.ext4 /dev/sdb1

# 3. Mount and restore data
sudo mount /dev/sdb1 /mnt/source
sudo rsync -av /backup/ /mnt/source/

Q: How do I optimize filesystem performance?

A: Several optimization techniques:

# For SSDs - enable TRIM
sudo mount -o discard /dev/sdb1 /mnt/data

# Disable access time updates
sudo mount -o noatime /dev/sdb1 /mnt/data

# For databases - use direct I/O
sudo mount -o barrier=0 /dev/sdb1 /mnt/database

# Optimize ext4 for SSDs
sudo tune2fs -o discard /dev/sdb1

Q: How do I recover deleted files?

A: Recovery methods depend on the filesystem:

# Install recovery tools
sudo apt install testdisk photorec extundelete

# For ext4 filesystems
sudo extundelete /dev/sdb1 --restore-all

# General purpose recovery
sudo photorec /dev/sdb1

# For immediate action after deletion
sudo grep -a -B25 -A25 'text from deleted file' /dev/sdb1

Coding Examples

Python Filesystem Analyzer

#!/usr/bin/env python3
"""
Filesystem analyzer for Ubuntu 22.04
"""
import os
import subprocess
import json
import time
from pathlib import Path

class FilesystemAnalyzer:
    def __init__(self):
        self.mountpoints = self.get_mountpoints()

    def get_mountpoints(self):
        """Get all mounted filesystems"""
        mountpoints = []
        try:
            with open('/proc/mounts', 'r') as f:
                for line in f:
                    parts = line.strip().split()
                    if len(parts) >= 3 and parts[0].startswith('/dev/'):
                        mountpoints.append({
                            'device': parts[0],
                            'mountpoint': parts[1],
                            'filesystem': parts[2],
                            'options': parts[3]
                        })
        except Exception as e:
            print(f"Error reading mount points: {e}")

        return mountpoints

    def analyze_filesystem(self, path='/'):
        """Analyze filesystem usage and characteristics"""
        try:
            # Get basic filesystem statistics
            statvfs = os.statvfs(path)

            total_size = statvfs.f_frsize * statvfs.f_blocks
            free_size = statvfs.f_frsize * statvfs.f_bavail
            used_size = total_size - free_size

            # Get inode information
            total_inodes = statvfs.f_files
            free_inodes = statvfs.f_favail
            used_inodes = total_inodes - free_inodes

            return {
                'path': path,
                'total_size_gb': round(total_size / (1024**3), 2),
                'used_size_gb': round(used_size / (1024**3), 2),
                'free_size_gb': round(free_size / (1024**3), 2),
                'usage_percent': round((used_size / total_size) * 100, 2),
                'total_inodes': total_inodes,
                'used_inodes': used_inodes,
                'free_inodes': free_inodes,
                'inode_usage_percent': round((used_inodes / total_inodes) * 100, 2) if total_inodes > 0 else 0
            }
        except Exception as e:
            return {'error': str(e)}

    def get_largest_files(self, path='/', min_size_mb=100, count=10):
        """Find largest files in a directory tree"""
        large_files = []
        min_size_bytes = min_size_mb * 1024 * 1024

        try:
            for root, dirs, files in os.walk(path):
                # Skip certain directories to avoid permission errors
                dirs[:] = [d for d in dirs if not d.startswith('.') and d not in ['proc', 'sys', 'dev']]

                for file in files:
                    try:
                        file_path = os.path.join(root, file)
                        file_size = os.path.getsize(file_path)

                        if file_size >= min_size_bytes:
                            large_files.append({
                                'path': file_path,
                                'size_mb': round(file_size / (1024**2), 2),
                                'size_gb': round(file_size / (1024**3), 2)
                            })
                    except (OSError, IOError):
                        continue  # Skip files we can't access

            # Sort by size and return top files
            large_files.sort(key=lambda x: x['size_mb'], reverse=True)
            return large_files[:count]

        except Exception as e:
            return [{'error': str(e)}]

    def get_directory_sizes(self, path='/', max_depth=2):
        """Get sizes of directories"""
        directory_sizes = []

        try:
            result = subprocess.run(['du', '-h', f'--max-depth={max_depth}', path],
                                  capture_output=True, text=True)

            for line in result.stdout.strip().split('\n'):
                if line:
                    parts = line.split('\t')
                    if len(parts) == 2:
                        size, dir_path = parts
                        directory_sizes.append({
                            'path': dir_path,
                            'size': size
                        })

            return directory_sizes

        except Exception as e:
            return [{'error': str(e)}]

    def check_filesystem_health(self, device):
        """Check filesystem health using fsck"""
        try:
            # Run read-only check
            result = subprocess.run(['fsck', '-n', device],
                                  capture_output=True, text=True)

            return {
                'device': device,
                'status': 'clean' if result.returncode == 0 else 'errors_found',
                'output': result.stdout,
                'errors': result.stderr
            }
        except Exception as e:
            return {'device': device, 'error': str(e)}

    def monitor_filesystem_performance(self, duration=10):
        """Monitor filesystem I/O performance"""
        try:
            # Start iostat monitoring
            process = subprocess.Popen(['iostat', '-x', '1', str(duration)],
                                     stdout=subprocess.PIPE,
                                     stderr=subprocess.PIPE,
                                     text=True)

            output, error = process.communicate()

            return {
                'duration': duration,
                'iostat_output': output,
                'error': error if error else None
            }
        except Exception as e:
            return {'error': str(e)}

    def generate_report(self):
        """Generate comprehensive filesystem report"""
        report = {
            'timestamp': time.strftime('%Y-%m-%d %H:%M:%S'),
            'hostname': os.uname().nodename,
            'mountpoints': self.mountpoints,
            'filesystem_analysis': [],
            'large_files': [],
            'directory_sizes': []
        }

        # Analyze each mounted filesystem
        for mount in self.mountpoints:
            analysis = self.analyze_filesystem(mount['mountpoint'])
            analysis['device'] = mount['device']
            analysis['filesystem_type'] = mount['filesystem']
            report['filesystem_analysis'].append(analysis)

        # Find large files in home and var directories
        for path in ['/home', '/var']:
            if os.path.exists(path):
                large_files = self.get_largest_files(path, min_size_mb=50, count=5)
                report['large_files'].extend(large_files)

        # Get directory sizes for common directories
        for path in ['/', '/home', '/var', '/usr']:
            if os.path.exists(path):
                dir_sizes = self.get_directory_sizes(path, max_depth=2)
                report['directory_sizes'].extend(dir_sizes)

        return report

# Example usage
if __name__ == "__main__":
    analyzer = FilesystemAnalyzer()

    # Generate and display report
    report = analyzer.generate_report()
    print(json.dumps(report, indent=2))

    # Save report to file
    with open(f'/tmp/filesystem_report_{int(time.time())}.json', 'w') as f:
        json.dump(report, f, indent=2)

Bash Filesystem Management Script

#!/bin/bash
# filesystem_manager.sh - Comprehensive filesystem management for Ubuntu 22.04

# Configuration
BACKUP_DIR="/backup"
LOG_FILE="/var/log/filesystem_manager.log"

# Colors
RED=''
GREEN=''
YELLOW=''
BLUE=''
NC=''

# Logging function
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${RED}This script must be run as root${NC}"
        exit 1
    fi
}

# Display filesystem information
show_filesystem_info() {
    echo -e "${BLUE}=== Filesystem Information ===${NC}"

    echo "Mounted Filesystems:"
    df -hT | grep -v tmpfs
    echo ""

    echo "Filesystem Types:"
    lsblk -f
    echo ""

    echo "Mount Points:"
    mount | grep "^/dev" | column -t
    echo ""
}

# Create filesystem with optimal settings
create_filesystem() {
    local device="$1"
    local fstype="$2"
    local label="$3"

    if [ -z "$device" ] || [ -z "$fstype" ]; then
        echo "Usage: create_filesystem <device> <filesystem_type> [label]"
        echo "Supported types: ext4, xfs, btrfs"
        return 1
    fi

    echo -e "${YELLOW}Creating $fstype filesystem on $device${NC}"

    case "$fstype" in
        "ext4")
            if [ -n "$label" ]; then
                mkfs.ext4 -L "$label" -m 1 "$device"
            else
                mkfs.ext4 -m 1 "$device"
            fi

            # Optimize for SSD if detected
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            if [ -n "$label" ]; then
                mkfs.xfs -f -L "$label" "$device"
            else
                mkfs.xfs -f "$device"
            fi
            ;;

        "btrfs")
            if [ -n "$label" ]; then
                mkfs.btrfs -f -L "$label" "$device"
            else
                mkfs.btrfs -f "$device"
            fi
            ;;

        *)
            echo -e "${RED}Unsupported filesystem type: $fstype${NC}"
            return 1
            ;;
    esac

    log_message "Created $fstype filesystem on $device"
    echo -e "${GREEN}Filesystem created successfully${NC}"
}

# Mount filesystem with optimal options
mount_filesystem() {
    local device="$1"
    local mountpoint="$2"
    local fstype="$3"

    if [ -z "$device" ] || [ -z "$mountpoint" ]; then
        echo "Usage: mount_filesystem <device> <mountpoint> [filesystem_type]"
        return 1
    fi

    # Create mountpoint if it doesn't exist
    mkdir -p "$mountpoint"

    # Detect filesystem type if not provided
    if [ -z "$fstype" ]; then
        fstype=$(blkid -o value -s TYPE "$device")
    fi

    # Set optimal mount options based on filesystem type
    case "$fstype" in
        "ext4")
            mount_options="defaults,noatime"
            # Add discard for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                mount_options="$mount_options,discard"
            fi
            ;;
        "xfs")
            mount_options="defaults,noatime"
            ;;
        "btrfs")
            mount_options="defaults,noatime,compress=zstd"
            ;;
        *)
            mount_options="defaults"
            ;;
    esac

    echo -e "${YELLOW}Mounting $device at $mountpoint with options: $mount_options${NC}"

    if mount -o "$mount_options" "$device" "$mountpoint"; then
        log_message "Mounted $device at $mountpoint"
        echo -e "${GREEN}Filesystem mounted successfully${NC}"

        # Add to fstab for permanent mounting
        uuid=$(blkid -o value -s UUID "$device")
        if [ -n "$uuid" ]; then
            echo "UUID=$uuid $mountpoint $fstype $mount_options 0 2" >> /etc/fstab
            echo "Added to /etc/fstab for automatic mounting"
        fi
    else
        echo -e "${RED}Failed to mount filesystem${NC}"
        return 1
    fi
}

# Check filesystem health
check_filesystem_health() {
    local device="$1"

    if [ -z "$device" ]; then
        echo "Usage: check_filesystem_health <device>"
        return 1
    fi

    echo -e "${BLUE}=== Checking filesystem health for $device ===${NC}"

    # Get filesystem type
    fstype=$(blkid -o value -s TYPE "$device")

    case "$fstype" in
        "ext4"|"ext3"|"ext2")
            echo "Running fsck for ext filesystem..."
            fsck.ext4 -n "$device"
            ;;
        "xfs")
            echo "Running xfs_check..."
            xfs_check "$device"
            ;;
        "btrfs")
            echo "Running btrfs check..."
            btrfs check "$device"
            ;;
        *)
            echo "Running generic fsck..."
            fsck -n "$device"
            ;;
    esac

    log_message "Filesystem health check completed for $device"
}

# Optimize filesystem performance
optimize_filesystem() {
    local mountpoint="$1"

    if [ -z "$mountpoint" ]; then
        echo "Usage: optimize_filesystem <mountpoint>"
        return 1
    fi

    echo -e "${BLUE}=== Optimizing filesystem at $mountpoint ===${NC}"

    # Get device and filesystem type
    device=$(df "$mountpoint" | tail -1 | awk '{print $1}')
    fstype=$(df -T "$mountpoint" | tail -1 | awk '{print $2}')

    case "$fstype" in
        "ext4")
            echo "Optimizing ext4 filesystem..."

            # Defragment if needed
            e4defrag -c "$mountpoint"

            # Optimize inode allocation
            tune2fs -o journal_data_writeback "$device"

            # Enable TRIM for SSDs
            if [[ $(cat /sys/block/$(basename $device | sed 's/[0-9]*$//')/queue/rotational) == "0" ]]; then
                tune2fs -o discard "$device"
                echo "SSD optimizations applied"
            fi
            ;;

        "xfs")
            echo "Optimizing XFS filesystem..."

            # XFS is generally well-optimized by default
            # Check if online defragmentation is needed
            xfs_fsr -v "$mountpoint"
            ;;

        "btrfs")
            echo "Optimizing Btrfs filesystem..."

            # Balance filesystem
            btrfs balance start "$mountpoint"

            # Defragment
            btrfs filesystem defragment -r "$mountpoint"
            ;;
    esac

    log_message "Filesystem optimization completed for $mountpoint"
    echo -e "${GREEN}Optimization completed${NC}"
}

# Backup filesystem
backup_filesystem() {
    local source="$1"
    local backup_name="$2"

    if [ -z "$source" ]; then
        echo "Usage: backup_filesystem <source_mountpoint> [backup_name]"
        return 1
    fi

    if [ -z "$backup_name" ]; then
        backup_name="filesystem_backup_$(date +%Y%m%d_%H%M%S)"
    fi

    backup_path="$BACKUP_DIR/$backup_name"

    echo -e "${YELLOW}Creating backup of $source to $backup_path${NC}"

    # Create backup directory
    mkdir -p "$backup_path"

    # Use rsync for efficient backup
    if rsync -avH --progress "$source/" "$backup_path/"; then
        log_message "Backup created: $source -> $backup_path"
        echo -e "${GREEN}Backup completed successfully${NC}"

        # Create metadata file
        cat > "$backup_path/.backup_info" << EOF
Source: $source
Backup Date: $(date)
Backup Size: $(du -sh "$backup_path" | cut -f1)
EOF
    else
        echo -e "${RED}Backup failed${NC}"
        return 1
    fi
}

# Monitor filesystem usage
monitor_filesystem() {
    echo -e "${BLUE}=== Filesystem Usage Monitor ===${NC}"

    while true; do
        clear
        echo "Filesystem Usage (updated every 5 seconds) - Press Ctrl+C to exit"
        echo "================================================================"

        df -h | grep -E "^Filesystem|^/dev"
        echo ""

        echo "I/O Statistics:"
        iostat -x 1 1 | tail -n +4
        echo ""

        echo "Top 5 largest directories in /:"
        du -h --max-depth=1 / 2>/dev/null | sort -hr | head -5

        sleep 5
    done
}

# Main menu
case "${1:-help}" in
    "info")
        show_filesystem_info
        ;;
    "create")
        check_root
        create_filesystem "$2" "$3" "$4"
        ;;
    "mount")
        check_root
        mount_filesystem "$2" "$3" "$4"
        ;;
    "check")
        check_root
        check_filesystem_health "$2"
        ;;
    "optimize")
        check_root
        optimize_filesystem "$2"
        ;;
    "backup")
        check_root
        backup_filesystem "$2" "$3"
        ;;
    "monitor")
        monitor_filesystem
        ;;
    "help"|*)
        echo "Filesystem Manager for Ubuntu 22.04"
        echo "Usage: $0 [command] [options]"
        echo ""
        echo "Commands:"
        echo "  info                          - Show filesystem information"
        echo "  create <device> <type> [label] - Create filesystem"
        echo "  mount <device> <mountpoint>   - Mount filesystem with optimal options"
        echo "  check <device>                - Check filesystem health"
        echo "  optimize <mountpoint>         - Optimize filesystem performance"
        echo "  backup <source> [name]        - Backup filesystem"
        echo "  monitor                       - Monitor filesystem usage"
        echo ""
        echo "Supported filesystem types: ext4, xfs, btrfs"
        echo ""
        echo "Examples:"
        echo "  $0 create /dev/sdb1 ext4 MyData"
        echo "  $0 mount /dev/sdb1 /mnt/data"
        echo "  $0 check /dev/sdb1"
        echo "  $0 optimize /home"
        echo "  $0 backup /home home_backup"
        ;;
esac

Best Practices for File Systems

Choosing the Right Filesystem

Decision Matrix:

Use Case                    | Recommended Filesystem
----------------------------|----------------------
General desktop/server      | ext4
Large files/databases       | XFS
Snapshots/modern features   | Btrfs
Maximum data integrity      | ZFS
Cross-platform compatibility| FAT32/NTFS
High-performance computing  | XFS or ext4

Performance Optimization

1. Mount Options:

   # For general use
   mount -o defaults,noatime /dev/sdb1 /mnt/data
   
   # For SSDs
   mount -o defaults,noatime,discard /dev/sdb1 /mnt/data
   
   # For databases
   mount -o defaults,noatime,barrier=0 /dev/sdb1 /mnt/database
   

2. I/O Scheduler:

   # For SSDs
   echo none > /sys/block/sda/queue/scheduler
   
   # For HDDs
   echo mq-deadline > /sys/block/sda/queue/scheduler
   

3. Filesystem Tuning:

   # Optimize ext4 for SSDs
   tune2fs -o discard /dev/sda1
   
   # Reduce reserved space (ext4)
   tune2fs -m 1 /dev/sda1
   

Security and Maintenance

1. Regular Checks:

   # Schedule regular filesystem checks
   echo "0 3 * * 0 /sbin/fsck -A -f" >> /etc/crontab
   

2. Backup Strategy:

   # Automated backups
   rsync -avH /home/ /backup/home/
   
   # Filesystem snapshots (Btrfs)
   btrfs subvolume snapshot /home /home/.snapshots/$(date +%Y%m%d)
   

3. Monitoring:

   # Set up disk usage alerts
   echo "df -h | awk 'NR>1 {if(\$5+0 > 85) print \$0}' | mail -s 'Disk Alert' admin@example.com" | crontab -
   

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training

| Wipro NGA Datacentre Training